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DYNAMIC SCALABLE FULL - PARALLAX 
THREE-DIMENSIONAL ELECTRONIC DISPLAY 

RELATED PATENT APPLICATION 

This application claims the benefit of U.S. 
Provisional Application No. 60/111,906, filed December 
10, 1998 and entitled "Dynamic Scalable Full-Parallax 
Three-Dimensional Electronic Display" . 

TECHNICAL FIELD OF THE INVENTION 

The present invention relates in general to large 
scale autostereoscopic displays and, more particularly, 
to a system and method which uses lenslet pixel modules 
to provide a dynamic, scalable, full -parallax three- 
dimensional electronic display. 
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BACKGROUND OF THE INVENTION 

The use of computers to assist in three-dimensional 
(3D) information design, visualization, and analysis is 
revolutionizing the communication of 3D information, 
especially through the emergence of the "virtual reality" 
paradigm. At present people can interactively experience 
3D information using a computer interface which usually 
requires wearing special glasses or a helmet 
incorporating goggles. These kinds of solutions are 
problematic in their unnatural obtrusiveness and 
difficulty in simultaneous participation of a group or 
large audience. 

Fly's eye lens sheets and related techniques were 
developed decades ago using optics and photographic 
technology. Fly's eye lens sheets possess many desirable 
qualities for use in 3D display such as autostereoscopy , 
high-resolution, full-color, high light efficiency 
(brightness) , full -parallax (both horizontal and vertical 
depth information) , and wide viewing angle (up to about 
forty-five (45) degrees) . Such displays often include a 
fly's eye lens sheet having a two-dimensional array of 
low f-number lenslets which are positioned in front of 
one or more two-dimensional (2D) images. The 2D image or 
images are projected into space by the fly's eye lens 
sheet. The sum of all rays of light projected from the 
lenslets approximates the directions and intensities of 
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light rays coming from a real 3D object or scene 
corresponding with the 2D image or images. 

Fly's eye lens sheets are often considered 
functionally analogous to lenticular lens sheets which 
are typically 2D arrays of semi-cylindrical lenslets used 
in creating 3D images with horizontal parallax only. The 
resulting lenticular displays have enjoyed more 
popularity than displays using fly's eye lens sheets 
primarily because lenticular lens sheets are technically 
easier to create. The application of lens sheet 
techniques to electronic displays is still in an early 
development stage. Typically, only lenticular lens sheet 
methods have even been commercially attempted, and only 
with relatively low resolution displays capable of pre- 
senting stereo- image pair views to an observer at a 
strictly proscribed viewing distance and head positions. 
Large 3D electronic displays having fly's eye lens sheets 
have generally not been considered due to associated 
costs and difficulty in manufacture and assembly of the 
display. 

There has been previous discussions about developing 
an optical input/output system having "projection pixels" 
with one or more "camera pixels" interleaved within the 
array of projection pixels. See for example "The I/O 
Bulb And The Luminous Room" prepared by John Stephen 
Underkof f ler published during February 1999 as partial 
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fulfillment of the requirements for a doctoral degree 
from Massachuset Institute of Technology. 



AUS01 : 187045 . 1 



ATTORNEY'S DOCKET 
065113 . 0116 



PATENT APPLICATION 



SUMMARY OF THE INVENTION 

In accordance with teachings of the present 
invention a system and method are provided to produce a 
scalable, 3D electronic display having a plurality of 
lenslet pixel modules. For some applications the display 
will produce a full parallax image. However, a 3D 
electronic display incorporating teaching of the present 
invention may be used to produce horizontal parallax only 
or vertical parallax only images. Each lenslet pixel 
module is designed in accordance with teaching of the 
present invention for optimal viewing characteristics 
such as wide output balanced with practical 
characteristics such as robustness and ease of 
manufacture. For one application, a large electronic 
display may be formed with as many of lenslet pixel 
modules as necessary to satisfy a given size or element 
resolution requirement. Fly ! s eye lens sheets may be 
used to form part of each lenslet pixel module. The 
modular approach of the present invention has many 
benefits related to image quality, flexibility, and 
reduced cost . 

One aspect of the present invention preferably 
includes using image display software within an 
electronic display to provide an independent 3D digital 
moving image transmission format along with a central 
control computer for producing and distributing 
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appropriate image information to associated lenslet pixel 
modules. In addition, a sensor such as a charge coupled 
device (CCD) can be incorporated into some or all of the 
lenslet pixel modules so that the resulting 3D electronic 
5 display may function as a 3D camera as well as a 3D 

display. For other applications, the 3D display may 
O include an array of sensor elements interspersed with an 

jbs. 
; a s 

S. array of lenslet pixel modules as opposed to combining 

sensor elements within respective lenslet pixel modules. 

Q 10 For still other applications, there may be a combination 

of sensor/projector elements, sensor-only elements and 
projector-only elements as desired for each specific 

•jf electronic display. 

ifi Technical benefits of the present invention include 

:j=j 15 providing autostereoscopic displays which are tailored to 

the human visual system in characteristics such as 
resolution and ergonomic ease of use. Technical benefits 
of such autostereoscopic displays also include presenting 
3D information to an individual or group of observers 
20 using computer mediated 3D communications in accordance 

with teachings of the present invention without requiring 
each observer to wear special goggles or glasses. 

Forming an electronic display using one or more 
fly f s eye lens sheets in accordance with teachings of the 
25 present invention provides vertical parallax information 

which may be very important when it is desirable to view 
an image from a variety of distances without distortions 
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and when it is desirable to change an observer's vertical 
position relative to the display in order to view the 
associated image from different vertical positions. 
Fly's eye lens sheet designs are generally more scalable 
in resolution and size due to the inherently symmetrical 
and compact lenslets. Fly's eye lens sheets generally 
produce a more realistic, less distorting and easier to 
view 3D image as compared to a lenticular type display. 

The present invention allows using high resolution 
2D image sources such as miniature cathode ray tubes 

(CRTS) , liquid crystal displays (LCDs) , digital micro 
device (DMD) mirrors, microelectronicmechanical systems 

(MEMS) and charge coupled device (CCD) sensors to provide 
a 2D image at each lenslet pixel module of a 3D 
electronic display at a reasonable cost. A wide variety 
of light valves and/or light modulators may be 
satisfactorily used to provide the desired 2D image 
preferably in a digital format to each lenslet pixel 
module. Some of these images sources may be low 
resolution . 

Computer control systems and software may be used to 
provide an image production and distribution system which 
presents the desired 2D image behind each lenslet pixel 
module at a desired frame rate. Miniature CRTS, LCDs, 
CCD sensors, and other high resolution light valves and 
light modulators have all recently reached an adequate 
level of miniaturization and developed into commodities 
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with costs decreasing at a constant rate such that a 
fly's eye lens sheet and multiple high resolution 2D 
image sources may be combined to form a dynamic 3D image 
display. 

Other technical benefits and advantages will be 
apparent to one of ordinary skill in the art after 
reviewing the specification, drawings and claims. 
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BRIEF DESCRIPTION OF THE DRAWINGS 

A more complete understanding of the present 
invention and advantages thereof may be acquired by- 
reviewing the following descriptions taken in conjunction 
with the accompanying drawings, in which like reference 
numbers indicate like features, and wherein: 

FIGURE 1 is a schematic drawing showing an isometric 
view of a portion of a parallax barrier screen display; 

FIGURE 2 is a schematic drawing showing a plan view 
of the parallax barrier screen display of FIGURE 1 ; 

FIGURE 3 is a schematic drawing showing an isometric 
view of a portion of a lenticular screen display; 

FIGURE 4 is a schematic drawing showing a plan view 
of the lenticular screen display of FIGURE 3; 

FIGURE 5 is a schematic drawing showing a plan view 
of another lenticular screen display having a first 
lenticular lens sheet and a second lenticular lens sheet 
with a diffused surface disposed therebetween; 

FIGURE 6 is a schematic drawing showing an isometric 
view of a display screen formed in part by a fly's eye 
lens sheet; 

FIGURE 7 is a schematic drawing showing a plan view 
of the display screen of FIGURE 6; 

FIGURE 8 is a schematic drawing showing a side view 
of a fly f s eye lens sheet used to record an image on 
photosensitive recording material; 
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FIGURE 9 is a schematic drawing with portions broken 
away of a single lenslet pixel module incorporating 
teachings of the present invention for use in a 3D 
electronic display; 

FIGURE 10 is a schematic drawing with portions 
broken away of a display system having a multiple lenslet 
pixel modules incorporating teachings of the present 
invention; 

FIGURE 11 is a schematic drawing with portions 
broken away of another display system having multiple 
lenslet pixel modules incorporating teachings of the 
present invention; 

FIGURE 12 is a schematic drawing showing a plan view 
with portions broken away of a lenslet pixel module 
having both a projector element and a sensor element; 

FIGURE 13 is a schematic drawing with portions 
broken away showing still another display system having a 
selected number of lenslet pixel modules with respective 
projector elements and a selected number of lenslet pixel 
modules with respective sensor elements in accordance 
with teachings with the present inventions; 

FIGURE 14 is a schematic drawing with portions 
broken away of a display system having multiple lenslet 
pixel modules which include both projector elements and 
sensor elements incorporating teachings of the present 
invention; and 
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FIGURE 15 is a schematic drawing with portions 
broken away showing still another display system having a 
selected number of lenslet pixel modules with high 
resolution and low resolution sensor elements and a 
selected number of lenslet pixel modules with a high 
resolution image display elements and low resolution 
image display elements. 
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DETAILED DESCRIPTION OF THE INVENTION 

Preferred embodiments of the present invention and 
its advantages are best understood by reference to 
FIGURES 1-15 of the drawings, like numerals being used 
5 for like and corresponding parts of the various drawings. 

The following written description discusses various 
embodiments of a scalable, full -parallax 3D electronic 
display formed in accordance with teachings of the 
present invention. The description* begins with a brief 
10 explanation of two similar technologies for providing 

autostereoscopic 3D image displays: parallax barrier 
displays and lenticular lens sheet displays. The 
description includes a comparison of lenticular lens 
sheet displays with fly's eye lens sheet displays and the 
15 benefits resulting using a fly's eye lens sheet to form a 

display in accordance with teachings of the present 
invention. A single lenslet pixel module having a fly's 
eye lens is described, followed by a discussion of some 
parallel signal distribution techniques to enhance the 
20 resulting 3D electronic display. The description 

concludes with examples of optional image acquisition 
hardware and software modifications. 

Pixel is an abbreviation for picture element. For 
purposes of describing various features of the present 
25 invent ion^-t-he-^t^rm "3D pixel" will be used to refer to 

illeat par^Vc 



of an electronically coded 3D graphic 




image. Pixel may also be used to refer to 
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the smallest addressable element in an electronic 
display. Conventional LCD Projectors and CCD sensors 
typically have an array of 2D pixels. The present 
invention provides an electronic display which produces a 
3D image having 3D pixels. 

The terms "lenslet pixel module" and " lenslet pixel 
modules" are used to describe various components such as 
high resolution and low resolution 2D image sources, 
fly ! s eye lenslet s and/or a sensor elements which may be 
combined in accordance with teachings of the present 
invention to form a 3D pixel in a resulting 
autostereoscopic image and/or detect a portion of a real 
3D object. Various features of the present invention may 
be described with respect to "projector pixels" which 
form a portion of an electronic display and "sensor 
pixels" which detect electromagnetic radiation from one 
or more real 3D objects or from a scene adjacent to the 
electronic display . 

The terms "projector module" or "projector modules" 
are used to refer to lenslet pixel modules which form a 
3D pixel in a resulting autostereoscopic image. The 
terms "sensor module", "detector module", "sensor 
modules" and "detector modules" are used to describe 
lenslet pixel modules which 'are responsive to 
electromagnetic radiation from a real 3D object or a real 
scene adjacent to an electronic display incorporating 
teachings of the present invention. 
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The term "high resolution" is used to refer to a 2D 
image source and to a sensor element having a 2D array of 
at least 1280 by 1024 pixels. 



Various Vembodiments of the present invention will be 
described with\respect to lens sheets and lenslets 
satisfactory forVse in forming 3D displays incorporating 
teachings of the p\esent invention may be formed from a 
wide variety of optimally transparent materials such as 
plastics and glass. Lens sheets and lenslets are often 
formed using injection molding techniques. Commercially 
available optical design and fabrication methods may be 
used to form various components of an electronic display 
incorporating teachings of the present invention. Also, 
the selection of various optical elements, their 
respective locations and optical characteristics such as 
aperture, diameter and focal length may be determined in 
accordance with commercially available\techniques 
associated with digital electronic and holographic 
displays . 



A basic characteristic distinguishing an 
autostereoscopic display from a 2D display is that an 
autostereoscopic display can generally control both light 
intensity and light directions emanating therefrom 
whereas a 2D display only modulates light intensity. A 
2D photograph or a 2D television set is perceived as a 
flat surface because the image surface diffuses light 
indiscriminately, whereas autostereoscopic displays 
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appear 3D because the observer's two eyes are presented 
with two different viewpoint -dependent images. The most 
natural kinds of autostereoscopic displays allow an 
observer to move around within a wide range of viewing 
positions, always presenting the correct information to 
the observer 1 s eyes . 

PARALLAX BARRIER DISPLAYS 

FIGURES 1 and 2 are schematic diagrams showing one 
example of a typical parallax barrier screen display 
indicated generally at 20. A parallax barrier screen 
display such as display 20 generally includes a 
transparency or sheet 22 with fine parallel vertical 
opaque lines 24 which are placed at a defined distance in 
front of a specially prepared picture or film 28 as shown 
in FIGURE 2. Sheet 22 and opaque lines 24 cooperate with 
each other to form parallax barrier screen 26. Specially 
prepared picture or film 28 disposed adjacent to parallax 
barrier screen 26 generally includes 2D images (not 
expressly shown) from different viewpoints which are 
positioned in fine strips behind lines 24 of parallax 
barrier screen 26 such that only the appropriate strips 
are visible from a particular viewing angle. 
FIGURE 2 is a diagram of a top down view of one 
embodiment of parallax barrier screen display 20. 

The parallax barrier screen technique has a set of 
advantages and disadvantages compared to alternative 
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techniques. Because of the offset printing industry's 
need for high-resolution, high-contrast black-and-white 
photographic transparencies used in producing printing 
plates, it is possible to inexpensively produce large 
5 raster barrier screens of any desired barrier pattern up 

to 2400 lines-per-inch. The essential drawbacks to 
raster barrier screen displays are in the unavoidable 
darkening of the image caused by the opaque barrier 
strips, and by the difficulties of registering the 
10 barrier screen with the specially prepared 2D image or 

images disposed adjacent thereto. The use of a lightbox 
(not expressly shown) to back illuminate a raster barrier 
screen can result in adequately bright images for certain 
viewing situations . 

15 

LENTICULAR SCREEN AND FLY'S EYE SCREEN DISPLAYS 

A lenticular screen display is geometrically similar 
to a parallax barrier screen display. The relationship 
between the two types of displays is analogous to the 
20 relationship of a pin-hole camera to a normal convex lens 

camera. A lenticular screen display is analogously much 
more light efficient than a parallax barrier screen 
display . 



FIGURES 3 and 4 are schematic diagrams showing one 
25 example of a lenticular screen ^display indicated 

generally at 3D. A principle advantage of a lenticular 
screen display such as display 3D Ys its superior optical 
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efficiency or brightnes 
display. 




to a parallax barrier 



For the example shpwn in FI^gRE5"~~3" and 4 lgrrficul-air 
screen display 3D preferably includes lenticular lens 
sheet 32 have a plurality \f parallel, vertical 
lenticules 34 form on one surface thereof. A specially 
prepared picture or film 38 may be disposed adjacent to 
lenticular lens sheet 32 opposite from lenticules 34 as 
shown in FIGURE 4. Picture or f ilfh 38 preferably includes 
10 one or more 2D images which will beXprojected from 

lenticular lens sheet 32 as a horizontal parallax only 3D 
image . 

Some disadvantages of a lenticular screen display 
are the difficulties and costs in designing and producing 

15 a good lenticular lens sheet with minimal aberrations. 

Another disadvantage is the problem of displaying 
lenticular images in outdoor situations where sunlight 
may be concentrated by the associated lenticules on the 
2D image plane behind the lenticules and cause damage to 

20 the picture or film having the 2D image. Both parallax 

barrier screen displays and lenticular screen displays 
unavoidably suffer from diffraction limited resolution 
related to the width of the parallax barrier slits or the 
lenticules in the case of a lenticular screen display. 

25 In order to display moving images, the static 2D 

image behind the parallax barrier screen or behind the 
lenticular screen may be replaced by a moving image 
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screen or moving image source with specially prepared 
images composed of fine vertical strip images carefully 
matched to the geometrical design of the parallax barrier 
screen or lenticular screen. It is possible to generate 
5 these special moving images by real-time parallel 

computer processing of conventional video output from a 
q line of video cameras (not expressly shown) or from syn- 

12 thesized views of a computer graphics animation rendering 

I*! engine (not expressly shown) . Alternatively, it is 

Q 10 possible to optically process and combine rear-projected 

Ijj images using two or more projectors 40 and 41 along with 

lenticular lens sheets 42 and 43 with diffusing surface 
44 disposed therebetween as shown in FIGURE 5 . 
i¥j Parallax barrier screen display 2 0 and lenticular 

15 screen display 3D are horizontal parallax only displays. 

These displays may be modified\to full parallax displays 



by replacing parallax barrier sheet 22 or lenticular lens 
sheet 32 with a pinhole array and a\ fly's eye lens sheet 
respectively . 

2 0 FIGURES 6 and 7 are schematic drawings showing one 

example of a typical fly's eye screen display shown 
generally at 50. A fly's eye screen display such as 
display 50 generally includes a fly's eye lens sheet 52 
having an array of fly's eye lenslets 54 disposed on one 

25 surface thereof. The opposite surface of a fly's eye 

lens sheet 52 is preferably flat and smooth. Fly's eye 
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lenslets 54 may be described as generally spherical 
bumps . 



Two-dimensional VLmage source 58 is preferably 
disposed on fly's eye \ens sheet 52 opposite from 
5 lenslets 54. Two-dimensional image source 58 may be a 

picture or film as previously described with respect to 
lenticular screen display 3D.\ For some applications 2D 
image source 58 may be a moving \j. mage screen or other 
moving image source 
10 Full parallax ^displays such as display 50 provide 

more realistic 3D images as compared to horizontal 
parallax only displaya such as displays 20 and 3D. The 
difference is most apparent when an observer moves in a 
vertical direction relatVve to the displays. Another 
15 benefit of a full parallaA display is in the freedom it 

allows an observer to view ^ display from any distance 
without observing the anamorihic distortions which are 
inherent in horizontal parallax only displays. The 
reason for these anamorphic distortions in lenticular 
20 displays such as display 3D is tVie fact that the vertical 

dimension of the images behind leYiticules 34 is fixed. 
The relative sizes of objects in tine real world change 
with the viewing distance. Since tke vertical dimension 
of objects in lenticular display 3D Is fixed, there can 
25 only be one viewing distance for whicm the vertical 

dimensions of the resulting 3D image a^e correct . 
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Image information can appear both in front and 
behind fly's eye screen display 50 as shown in FIGURE 7. 
FIGURE 7 shows a cross-section of fly's eye screen 
display 50 where point A is imaged in front of display 50 
5 and point B appears to be behind display 50 simply due to 

the apparent convergence of the rays projected by fly's 
eye lenslets 54 . Image points which are recorded on 2D 
image source 58 disposed behind lenslets 54 are refracted 
by lenslets 54 and emerge in directions consistent with 

10 light from a corresponding real 3D object or scene. 

One method for creating synthetic 2D images for a 
fly's eye screen display can be inferred from FIGURE 7, 
namely computer graphic ray tracing. Unfortunately, 
creating real-world images is more complicated due to a 

15 peculiar depth inversion problem which is briefly 

described below. 



\Q FIGURE 8 depicts \^ side view of a model of a cat 

/S^ placed in front of fly'^eye lens sheet 52. For purposes 

of explanation a layer of\photosensitive recording 

20 material may be placed on lens sheet 52 at the surface 

previously occupied by 2D soVrce image 58 . Lenslets 52 
appear to image the cat as ona would expect from the laws 
of optics, namely a small inverted image of the cat from 
the perspective of each lenslet i^s imaged behind it on 

25 the photosensitive recording material at the 2D image 

source plane. Since the cat is fac\ng fly's eye lens 
sheet 52, each lenslet 54 images a front view of the cat. 
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15 



20 



of fly's eye lens sheet 52. 

The peculiar problem becomes evident when an 
observer tries to view the image from the front . When 
observed with a single eye from any stationary position, 
the cat appears normal, but when observed with both eyes, 
the 3D image is observed to be inverted in depth relative 
to the observer, with the cat's nose farther from the 
observer than it's tail. This problem associated with 
lens-sheet recordings was first noted by H. E. Ives in 
"Optical Properties of a Lippman Lenticulated Sheet, " J. 
Opt. Soc. Amer. 21, March, 1931. 

Ives' solution to this problem was to re-record the 
depth- inverted lens-sheet image in order to correct the 
depth. This approach was effective in correcting this 
problem, but it introduced additional noise in 
photographic processes, second-generation optical 
aperture-sampling artifacts, and other noise and 
artifacts due to compounded lenslet aberrations and 
diffraction. 

In a digital implementation the solution can be much 
more elegant. Conventionally rendered computer graphics 
images or real -world images acquired with a digital 
camera (not expressly shown) or other suitable sensor can 
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be systematically inverted to correct the problem. The 
algorithm used can be a variation of an algorithm 
developed, which was the subject of a masters' degree 
thesis by M. Holzbach, "Three-dimensional Image 
Processing for Synthetic Holographic Stereograms," MIT 
Master's Thesis, September, 1986. 

LENSLET PIXEL MODULES 

There are several objective characteristics for 
evaluating 3D displays. One obvious characteristic is 
the 2D image plane pixel resolution which is simply the 
number of lenslet pixel modules in a horizontal and 
vertical direction, and their spacing. This resolution 
is all one eye can appreciate from a single position in 
front of the display. 

Another measure of image quality of 3D images is 
depth resolution which depends on factors such as lenslet 
size and quality and image resolution behind each lens- 
let. Depth resolution for a fly's eye screen is not 
constant regardless of distance from the screen. Due to 
physical optical limits such as diffraction and the 
effects of lens aberrations and image misregistrations 
(which increase at a distance) , depth resolution is best 
for points close to the screen (on the order of the 
lenslet size) and decreases with the distance from the 
screen. * Complete analysis of depth resolution during 
lenslet design phase is preferably conducted as part of 
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forming a 3D electronic display in accordance with 
teachings of the present invention to optimize lenslet 
pixel module design to yield the best possible depth 
resolution . 

5 The image information for a fly's eye screen display 

originates on the source image plane behind the lenslets. 
If the 2D image source plane is positioned at the focal 
length of the lenslets, a point on the 2D image source 
plane behind a lenslet becomes a collimated (parallel 

10 light) beam which appears to completely fill the lenslet 

when observed at the correct angle relative to the 
lenslet. When the observed angle relative to the lenslet 
changes slightly, a neighboring image point from the 2D 
image source plane appears to completely fill the 

15 lenslet. From this it can be seen how the size and den- 

sity of neighboring image points on the 2D image source 
plane are directly related to depth resolution. 

A high resolution 2D image source, such as a 
miniature CRT or LCD screen, behind each lenslet is often 

20 desirable to optimize the depth resolution. However, a 

less-costly lower resolution alternative may also be 
achieved by positioning multiple neighboring lenslets to 
share dedicated regions of a single CRT or LCD screen, or 
sections of a diffuser back-lit by a video projection 

25 system (not expressly shown) . As noted later in this 

written "description, a wide variety of light valves and 
light modulators may be satisfactorily used with lenslet 
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pixel modules incorporating teachings of the present 
invention. Although high resolution 2D image sources are 
preferred, a 3D electronic display may be formed in 
accordance with teachings of the present invention using 
5 low resolution 2D image sources. Also, a wide variety of 

electromagnetic sensor and detectors may be 
;5as satisfactorily used with lenslet pixel modules 

l S3 incorporating teachings of the present invention. 

\T\ If the 2D image source positioned directly in front 

q 10 of one lenslet is not obstructed from the aperture of a 

;7J neighboring lenslet, it is possible that light from the 

iE neighboring 2D image source will cross though it at a 

1=1= 

jtj steep angle. To an observer, the combined result when 

:5. viewing the entire display is a repeat or "flip" of the 

^ 15 3D image observed at angles greater than the originally 

intended viewing angle. This type of image "flip" can be 
considered a convenient way of allowing the display to be 
seen over an increased viewing angle, but it also 
distorts the image. Whether or not "flip" is considered 
2 0 an advantage or a disadvantage depends on the particular 

user application. If undesirable, "flip" can be 
eliminated by designing image blocking surfaces (not 
expressly shown) between the lenslets to block the 2D 
image source from neighboring lenslets from cross over. 
2 5 FIGURE 9 is a schematic diagram showing one 

embodiment of single lenslet pixel module 70 
incorporating teachings of the present invention. 
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Lenslet pixel module 70 preferably includes fly's eye 
lenslet 74 so that a 2D image may be projected from high 
resolution 2D image source 78 through fly's eye lenslet 
74. Other types of lenslets satisfactory for use in 
forming a portion of a 3D image may be used. The present 
invention is not limited to fly's eye lenslet 74. 
Lenslet pixel module 70 may sometimes be referred to as a 
projection module or a projection pixel . 



iff \Q Various types of comn^ercially available light valves 

^-H^XgLlO and light modulators may b<k satisfactorily used as high 



resolution 2D image source 7« . For the embodiment shown 
n in FIGURE 9, high resolution 2D image source 78 may be a 

jjJL \ 

jlj digital flat panel display (FPDO , an LCD or CRT. Other 

O \ 

r=Ps types of high resolution 2D imagle sources including light 

; S 15 emitting diodes (LED) . DMD mirrons and MEMS may also be 

satisfactorily used to form lensletX pixel modules 70. 

A plurality of lenslet pixel modules 70 may be 
arranged relative to each other in a generally 
rectangular array to provide an electronic 3D display 
20 (not expressly shown) having a resolution of 640 x 480 3D 

pixels. The front of such an array may look similar to 
display 50 in FIGURE 6. For some applications, fly's eye 
lenslet 74 may have a diameter less than one inch. 



A standard video source\ (not expressly shown) shown 
25 as NTSC or VGA ma Y be connected to high resolution 2D 



image source 78. For other applications, digital data 
may be supplied to high resolution 2D image source 78 
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which then converts"""fefcie digital data into the desired 2D 




r\j image. An important aspecfNif the present invention 



includes providing moving images eleven live images to 
lenslet pixel modules 70. 
5 A 3D electronic display may be formed in accordance 

with teachings of the present invention by using fly's 
eye lens sheet 52 as shown in FIGURE 6 to form a 
generally rectangular, flat array. For other 
applications lenslet pixel modules 70 may be used to form 

10 a generally curved array (not expressly shown) . Lenslet 

pixel modules incorporating teachings of the present 
invention may be arranged in a wide variety of arrays 
and/or mosaics as desired to provide the resulting full 
parallax 3D image. 

15 A plurality of high resolution 2D image sources may 

be coupled with respective fly's eye lenslet 54 to form 
the desired lenslet pixel module array. For other 
applications two or more lenslets 74 may share dedicated 
regions of the same high resolution 2D image source 78 

20 (not expressly shown) . For example a high resolution FPD 

may be combined with fly's eye lens sheet 52 in 
accordance with teachings of the present invention. 

FIGURE 10 is a schematic drawing showing system 80 
incorporating teachings of the present invention for 

25 presenting a scalable, full parallax autostereoscopic 

image. System 80 preferably includes a plurality of 
lenslet pixel modules 70 which may be arranged relative 
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to each other in a wide variety of arrays or mosaics. 
For one example lenslet pixel modules 7 0 may be arranged 
to form an array similar. to display 50 as shown in FIGURE 
6. For other applications lenslet pixel module 70 may be 
5 arranged in various geometric configurations such as 

concave, convex, rectangular, square or cylindrical. 

System 80 also includes a plurality of slave 
computer processing units (CPU) 82 with one or more 
lenslet pixel modules 70 coupled with each slave CPU 82. 

10 Depending upon the type of light valve or light modulator 

used to form lenslet pixel module 70, a video output 
channel may be used to couple each slave CPU 82 with its 
associated lenslet pixel modules 70. For other 
applications each slave CPU 82 may have a digital 

15 connection with its associated lenslet pixel module 70. 

Digital data from each slave CPU 82 may be converted to 
the desired 2D image by high resolution 2D image source 
78 . 

System 80 preferably includes master computer 
20 processing unit (CPU) 84 which is preferably connected 

with and controls the operation of each slave CPU 82 . 
For the embodiment shown in FIGURE 10, three lenslet 
pixel modules 70 are coupled with each slave CPU 82. 
However, the number of slave CPUs, master CPUs and 
25 lenslet pixel modules may be varied in accordance with 

teachings with the present inventions to provide the 
desired scalable, full parallax autostereoscopic image. 

AUS01 : 187045 . 1 



ATTORNEY'S DOCKET 
065113 . 0116 



PATENT APPLICATION 



28 

If the cost of providing an individual 2D image 
source for each lenslet is too expensive, a larger FPD 
(not expressly shown) or a video screen (not expressly 
shown) may be divided into four equal regions and shared 
among four lenslet pixel modules 70. The schematic 
illustration for this would be similar to FIGURE 10, only 
with each slave CPU 82 providing the desired 2D image 
input for four lenslets pixel modules 70 instead of three 
lenslet pixel modules 70. 

For some 3D electronic displays, each lenslet pixel 
module 70 may contain an internal CPU, a communications 
adapter, and video display circuitry so that each lenslet 
pixel module 7 0 may generate its own 2D image after being 
loaded with desired database and viewpoint information. 
FIGURE 11 is a schematic diagram showing system 90 having 
a plurality of lenslet pixel modules 70 wherein each high 
resolution 2D source 78 preferably includes a respective 
internal CPU (not expressly shown) , a communications 
adapter (not expressly shown) and associated video 
display circuitry (not expressly shown) along with the 
desired data base and viewpoint information. Master CPU 
94 provides the desired digital data input to each 
lenslet pixel module 70. Consumer video game player 
systems which are available in compact sizes and at 
relatively low cost may be the basis for the electronic 
circuits and components for lenslet pixel modules 70. 
The benefits of this massively parallel computer 
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processing approach would be in distribution of the 
computation load, relaxation of signal distribution 
requirements, and greatly simplified connectivity as 
shown in FIGURE 11. 

LENSLET PIXEL MODULES WITH SENSORS OR DETECTORS 

Inclusion of a sensor or detector responsive to 
electromagnetic radiation from a real 3D object or scene 
within one or more lenslet pixel modules 70 will open up 
many interesting capabilities and uses for the associated 
full-parallax 3D display. As discussed later with 
respect to FIGURE 12 , a CCD sensor or a miniature video 
camera may be incorporated into a lenslet pixel module 
having a partially silvered mirror and additional optics 
to virtually superimpose a 3D output image and a 2D 
sensor array. A consequence of this approach may be 
output brightness attenuation. Also, possibly increased 
bulk of the CCD sensor may require pixel module 
engineering to optimize lenslet pixel module packing 
density . 

The integration of a 2D sensor array into a 3D 
electronic display in accordance with teachings of the 
present invention will allow full -parallax 3D image 
information of real 3D objects, scenes, and people to be 
easily acquired. This rich level of 3D data has many 
potential military and civilian applications such as 
object measurement, virtual reality simulation source 
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imagery, medical diagnostic information acquisition, and 
damage assessment information to name a few. Because the 
information could be acquired as moving images in 
real-time, the 3D data could be used as the basis for 
virtual -reality computer-human interaction. For example, 
multiple human observers could have simulated 
computer-graphic objects tracking their moving bodies in 
real time. Perhaps some observers could have their 
images appear on the associated 3D display in real time, 
while other observers choose to remain invisible. 

Figure 12 is a schematic drawing showing lenslet 
pixel module 170 with projection element 78 and sensor 
element 172 combined in accordance with teachings of the 
present invention. Lenslet pixel module 170 preferably 
includes both high resolution 2D image source 78 and 
sensor element or detector element 172. Partially 
silvered mirror 176 is preferably provided to optically 
couple both high resolution 2D image source 78 and sensor 
element 172 with lenslet 74 . A wide variety of sensors 
or detectors which respond to electromagnetic radiation 
from a real 3D object or scene may be satisfactorily used 
as sensor element 172. 

For some applications high resolution 2D image 
source 7 8 may be coupled with a source of moving video 
images. Sensor element 172 may be coupled with an 
appropriate recorder or a computer processing unit to 
record images corresponding with one or more real 3D 
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objects (not expressly shown) disposed in front of 
lenslet 74 . 

Figure 13 is a schematic drawing showing various 
portions of system 180 which may be satisfactorily used 
5 to form an autostereoscopic image in accordance with 

teachings of the present invention. System 180 
preferably includes a plurality of lenslet pixel modules 
70 and a plurality of lenslet pixel modules 170. For 
purposes of describing various features of the present 
10 invention lenslet pixel modules 70 may also be referred 

to as projector modules. Lenslet pixel modules 170 may 
be described as combined pro j ector/sensor modules. 
lj Master CPU 94 is preferably provided to communicate 

2 digital data corresponding with a desired 2D image to 

3 15 both lenslet pixel modules 70 and lenslet pixel modules 

170. Master CPU 94 is also preferably operable to 
receive digital information from lenslet pixel modules 
170 which correspond with one or more objects disposed in 
front of the associated lenslets 74 . 

20 For some applications, system 180 may be formed from 

a plurality of lenslet pixel modules 70 having associated 
high resolution 2D image sources 78 or projector elements 
78 and a plurality of lenslet pixel modules or sensor 
modules having only associated sensor elements. One 

2 5 example of such sensor modules (not expressly shown) may 

be formed by replacing projector element 78 of lenslet 
pixel module 70 as shown in FIGURE 9 with a desired 
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sensor or detector element. System 180 may be 
satisfactorily formed with multiple projector modules, 
sensor modules and/or combined projector and sensor 
modules . 

5 FIGURE 14 is a schematic drawing showing system 190 

which may be used to form an interactive autostereoscopic 
^ display in accordance with teachings of the present 

f0 invention. System 190 preferably includes master CPU 94 

i Ik 

jjj coupled with a plurality of lenslet pixel modules 170. 

10 For this embodiment the resulting 3D array includes a 

1°^' plurality of lenslet pixel modules 170 having both a 

n projector element or (high resolution 2D image source 78) 

j|j and sensor element 172 . FIGURE 15 is a schematic drawing 

;;=f showing system 2 00 which may be used to form an 

''Q 15 interaction autostereoscopic display in accordance with 

~\ teachings of the present invention. System 200 

preferably includes master CPU 94 and a plurality of 
lenslet pixel modules 70 and 170. In addition, system 
200 also preferably includes a plurality of lenslet pixel 
20 modules 70a which have a low resolution 2D image source 

(not expressly shown) and a plurality of sensor of 
lenslet sensor modules 170a which preferably include a 
low resolution sensor (not expressly shown) . 

Various features of the present invention have been 
25 described with respect to fly's eye lenslet sheet 52, 

associated lenslets 54 and fly's eye lenslet 74. 
However, it is well known in the art that a wide variety 
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lens sheets and lenses may be satisfactorily used to form 
3D autostereoscopic images. Lens sheets and lenslets 
satisfactory for use in forming 3D electronic displays in 
accordance with teachings of the present invention may be 
fabricated using refractive methods or diffractive 
methods. A wide variety of lens sheets having a suitable 
lens array formed on a first surface and a second 
generally smooth flat surface disposed opposite from the 
first surface may be satisfactorily used to form lenslet 
pixel modules in accordance with the teachings of the 
present invention. The lenslets may be cylindrical, 
convex, concave, gradient index, diffractive, refractive, 
holographic optical elements or any other suitable prism 
which will form a full parallax 3D image. 

For some 3D displays a plurality of sensors or 
detectors (not expressly shown) may be dispersed within 
an array lenslet pixel modules having only projector 
elements. The lenslet pixel modules may cooperate with 
each other to form a projector array. The sensor 
elements may also cooperate with each other to form a 
sensor array. The lenslet pixel modules and each sensor 
may include a fly's eye lenslet. For some applications, 
the sensor array may have a focal plane which corresponds 
approximately with a focal plane associated with the 
projector array. For other applications the sensor array 
may have a focal plane oriented substantially different 
from the focal plane of the projector array. For still 
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other applications one or more of the sensors may have a 
lenslet with a configuration different from the projector 
array. Also, the lenslet pixel modules in the projector 
array may have different lenslets. The sensor array may 
5 cooperate with the projector array to allow an observer 

to interact with a 3D image formed by the projector 
array . 

The number of potential applications which could use 
a large 3D display incorporating teachings of the present 

10 invention is vast. It is expected that many people with 

virtual reality applications would be potentially 
interested in 3D electronic displays incorporating 
teachings of the present invention because of its unique 
aspects and capabilities. Some potential applications 

15 include mission planning, reconnaissance, damage 

assessment, scene analysis, education, training, 
simulations, mechanical and electrical engineering, 
cartography & terrain imaging, geoscience, seismic 
research, medical imagery, product design, architecture, 

20 space planning, advertising, scientific visualization and 

teleconferencing . 

Although the present invention and its advantages 
have been described in detail, it should be understood 
that various changes , substitutions and alterations may 

25 be made thereto without departing from the spirit and 

scope of the present invention as defined in the 
following claims. 
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